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The role of the professional evaluator within a 
school district is essentially to provide data for use in informed 
decision making. In School District 4J in Eugene, Oregon, this role 
involves performing tasks in three basic categories: (1) maintaining 
and interpreting ongoing databases, (2) finding and analyzing 
information to answer specific administrative questions, and (3) 
assisting with the district's cycle of program evaluation. The 
ongoing databases, distinct from the district's mainframe computer 
records, provide information for enrollment projection, high school 
subject area enrollment trends, and longitudinal analysis of 
standardized test results. Requests for special-purpose information 
can lead to quick responses or require major reports. Typical 
requests have sought information on the potential impact of changes 
in graduation requirements, on predictors of the tendency to drop out 
of school, on the effects of an open enrollment policy, and on space 
utilization in an elementary school. The district's subject area 
programs are evaluated every 6 years, with the evaluator assisting in 
evaluation design and gathering and summarizing information for the 
subject area council. As purveyors of data, the evaluators 
collaborate with other educators in the evaluation process, promoting 
cooperative and productive decision making. (PGD) 
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This symposium weis conceived during a late-night conversation at last year's AERA 
convention. The evaiuation department from Scbool District 4J. Eugene, Oregon (both of 
us) were talking about the day's events and how they related to our district. The conver- 
sation rambled, as such conversations do. ultimately leading us to the startling realization 
that research and evaluation in our district was a much broader undertaking than wo 
had previously thought. Indeed, when we considered the many forms of evalu^on. we 
realized that the role of the professional evaluator. at least in our district, had changed 
and evolved into something new. The purpose of this paper is to examine that role and to 
try to set a contest for the remaining papers in this symposium. 

The changing role of evaluation in District 4J has not yet developed to the point that 
it could be advanced as a flill*fledged evaluation model, so I will not deliver a treatise 
proposing one. I will instead describe the duties of the evaluation staff, within a very 
broad conceptual framework. That framework is best described by three general 
principles- 

1 . The evaluator's chief function is to provide data for informed decision-making. 
This is not to imply that the evaiuator's role is a purely passive one of gatheriiig data and 
allowing others to make decisions. Indeed, it is often necessary to prod decision-makers 
to pay attention to data they are ignoring. The Aindamental Amotion of the evaluator. 
however, remains the same ~ purveyor of information within a decision-making team. 

2. Information is useful only if the decision-makers understand it. Simplicity of 
analyses and clarity of writing are viied. Furthermore, iiiformation should not be 
restricted to quantitative data. Anecdotal data and qualitative information are often more 
easily understood and can be every bit a^ valid. Most important of all. data is less likely to 
be misunderstood if we have maintained clear and frank verbal communication with the 
users of our data. 

3. Evaluation is more likely to lead to positive action if it is cooperative than if it is 
adversarial. Evaluaiion will be more successful if those affected by its results are 



involved in the process. Evaluators are seen less as internal auditors and more as 
facilitators of self-examination and change. 

The above framework has resulted in part from a reorganization of the chain of 
command in the district. Whereas the evaluation unit was formerly an independent 
department, it is now under the supervision of the Assistant Superintendent for 
Instruction. It is conceivable that this might not always be an ideal arrangement, but in 
this case it is working well. In no way are the evaluators limited to instructional 
concerns, as we will see, nor are any constraints put on the oianner in which we are free 
to report data. Most important, this arrangement has been central in promoting 
cooperation and communicaiion betvreen evaluation and instructional staff. 

The evaluators' tasks can be classified into three broad categories - 

1. maintaining and interpreting ongoing databases, 

2. finding and aiial3^ing informiation to answer specific administrative questions, and 

3. assisting with the district's cycle of profjram evaluation. 

Oagomg datobftses 

There are two major types of databases with which we must deal - those housed in 
the district's mainframe computer records and those maintained by the evaluation 
department. The evaluation department is not responsible for mainframe computer 
records, nor do we have on-line access to them. These records enter into our work in 
many ways, but their maintenance can hardly be described as one of our tasks. The 
databases maintained by the evaluation department are a more direct concern. 

Enrollment projection is a rather straightforward example of the application of one 
of our ongoing databases. The projections are made in October for the following year, 
with occasional revisions during the spring. They are the basis for staffing, scheduling, 
and some parts of revenue projection. Several years ago. a district employee wrote a 



microcomputer program that would store five years' historical data and give the user the 
choice of five different projections of future enrollment. Once the total district 
projection is settled, individual school enrollments are estimated, using a slightly 
different database, namely, torenty years* worth of knowledge of the district. This method 
has been used with great success. Never has the overall projection differed from the 
actual district (enrollment by more than half of one percent and seldom are more than a 
handful of schools misprojected by more than five percent. 

Another database maintained by the evaluation def artment records high-school 
enrollments by subject area. The mainframe computer records have this in formation, 
but only for the current term. Furthermore, these records are awkward to use. For 
instance, there are three categories each for Science and Physical Education, but all Fine 
and Performing Arts. Industrial Arts, and Home Economics share a single category. For 
these rea^fis. we developed a database that shows subject-area enrollments in each high 
school going back to 1980-Sl. It shows some differences between schools, but more 
importantly, it shows a dramatic decline in elective credit in vocationally-related 
courts, a lesser decline in fine and performing arts, and increases in English, science, 
and foreign language credit. In most cases, these changes began at approximately the 
time that increased graduation requirements were first proposed or that the state 
university system proposed a change in entrance requirements. This database has 
allowed the district to avoid overprojecting effects of changing requirements when they 
actually come into effect. It has also raised concerns about the fate of the programs in 
decline. 

The largest single ongoing task of the evaluation department is the annual 
standardized testing program. Although this could easily be considered a fourth category 
of task. I include it as an ongoing database. The primary use of test data in this district is 
for longitudinal comparisons. We discourage school-to-school comparisons of test data. 
Unfortunfitely. there Is no system for computerized tracking of students' test scores, so 



our test data may be seriously underutilized. 

In addition to the databases maintained by the evaluation department and those 
housed on the mainframe computer, we regularly utilize Lane Cbunty's data system. This 
system includes demographic data, land-use data, and a mapping system. Annually we 
forward a computer tape of our complete student address file to the county, enabling 
them to provide us with maps of students' residences when needed (more of this later). 
Data from this system has also been used in long-range planning (e.g. school closings, 
sale of vacant buildings). 

Specific requests for iAforAstioA 

The second major category of task is to dig out and analyze information for specific 
purposes. These purposes vary from numerous small requests for information to a few 
major reports each year. The requests for information are intriguing and often require 
some ingenuity to flilfif 1. This category of task, while probably the most enjoyable, has 
its frustrations, either from data that cannot be verified or from seeing one's work being 
ignored in the final decision. These requests have one further advantage - they help to 
even out the evaluators' workloads. The major reports are seldom written on short 
timelines and can thus be left on the back burner, to be taken up again when time 
permits. 

A typical study of this kind resulted from a proposal by a district committee to 
increase graduation requirements even beyond those imposed by the state. This proposal 
would also have eliminated Plan IIL a local option whereby students can tailor their own 
graduation plans. We were asked to project the effects on students and schools of the new 
requirements. Ve drew a sample of one-quarter of the previous year's graduates and 
examined their transcripts. We discovered that, whereas the Plan III students had taken 
generally more rigorous academic programs, none of the sample would have met ail of 
the new requirements. The staffing changes needed would have taken at least one- 



quarter of the positioasin elective subjects (including foreign language, a growing 
area) and reassigned them to required subjects. The proposal has already been modified, 
but is still officially on the table. 

District 4J has very little information on its dropouts, so we conducted a study on 
dropouts from the class of '84. To identify the dropouts, ve had to vork in reverse. Ve 
compared the sophomore class lists from 1981-82 with the list of graduates from 1984, 
then tried to account for the remaining students. Those who were not still in our schools 
or could not be traced to another district or to the community college's high-school 
completion program, were considered to have dropped out. Three separate dropout rates 
were computed, based on three different definitions of dropout. A further analysis 
compared these students on several variables with a group that had graduated in 1984. 
The single strongest predictor of non-graduation in our variable set was Grade 9 
attendance. This was no surprise, but the strength of this prediction was remarkable. A 
discriminant analysis using several variables (including absences, test scores and such 
family data as could be found) classified 78 percent of the two groups correctly, but a 
simple break at 10 absences in Grade 9 would have classified 68 percent of each group 
correctly. That is, two-thirds of the graduates had 10 absences or fewer; two-thirds of the 
dropouts had more than 10. 

Two other examples indicate the resourcefulness that these information requests 
bring out. District 4J has an open enrollment policy which allows any student to enroll 
in any school in which there is room, provided they have an administrative transfer. 
Such transfers are routinely given. Allegations of abuses of this system resulted in a 
request to assess the effects of the policy. The county's mapping system gave us lists of 
students not attending their "home" schools, hut this was not enough. The writer had to 
try to verify that those attending their "home" school actually lived where they claimed 
to live. The county mapped the given addresses of students at each high school. Aside 
from one student having an address at a university dormitory and another allegedly 



living in a church parkiag lot, this method turned up very little. Ve finally tried 
cross-matching students* given addresses to those listed in the telephone directory, but 
found very few anomalies. 

In another case, several elementary schools complained of overcrowding and the 
r^l went out to us to assess the severity of the situatioa. Building plans and a ten-year- 
old report on school capacity convinced us that the problem.was not a lack of space, but 
simply too many rooms being dedicated to special uses. 

The cycle ofprogrAA evAlOAOion 

The final category of tasks is to assist with the ongoing cycle of program 
evaluation. In this district, all subject areas are on a six-year evaluation cycle. In those 
subjects with district-adopted textbooks, the evaluations precede textbook adoption by one 
year. The evaluations themselves are planned by the subject-area councils, who also are 
responsible for the final evaluation reports. Es^h council consists of teachers, building 
administrators, and one central-office administrator. The role of the professional 
evaluator in this evaluation cycle is twofold - to assist with the design of the evaluation 
and to gather and summarize information requested by the council. 

A loosely- constructed outline termed the AFE ( All-Purpose Evaluator) is often used 
to give the council a start in defining the issues of the evaluation. The APE outlines four 
possible areas on which the coun cil may wish to focus - 

1. Goal Verification. If -goals have not been re-examined recently, this should be the 
starting point. In District 4J, all subject councils have recently written official position 
papers, so goal verification has not been an issue in recent evaluations. 

2. Program Description. In several programs, there is little baseline data for 
comparisons. A detailed description of the current program will become baseline data for 
the next evaluation cycle. 

3. Implementation Verification. The one issue that the subject councils most want to 
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address is, "Hov veil does the actual program match that which appears on paper?" The 
questions asked here require much of the same data as program description, but involve 
comparisons to the standard spelled out in district documents. 

4. Assessment of Student Outcomes. In some subjects, such as Language Arts, there is 
ample data, on student outcomes. In others, very little data exists. Unfortunately, those 
tend to be the subjects in which there are very few instruments to assess student 
outcomes. 

Once the council has discussed the APE and set its priorities, evaluation staff collect 
and sumjnarize data for the final report. Quantitative data is not the only type included, 
of course. In the evaluation of art. social studies, and music programs, for instance, 
directed interviews with administrators and group discussions with teachers figured 
strongly. These directed discussions followed a set pattern. A list of questions for 
discussion was circulated ahead of time to the personnel involved. The group discussions 
were conducted with a tape recorder running (alter all present had consented) so that 
the interviewer would be free to participate in the discussion rather than take notes. The 
tape was later summarized in the relative quiet of the office. 

This approach to program evaluation, probably more than anything else, has 
contributed to the atmosphere of cooperation which I mentioned as the third point of the 
conceptual framework for our role. Teachers and administrators are realistic and blunt 
about the shortcomings of their programs. If the criticisms have come from them in the 
first place, they are overjoyed to see someone in central office take note. I doubt if they 
would be so excited were we to advance these criticisms on our own. 

ConclosioA 

The title of this symposium asks the question. "Who evaluates school programs?" As 
I indicated at the outset, this paper was not intended to advance a model for the role of the 
professional evaluates but simply to describe that role as it exists in one school district. 



The persons called 'professional evaluators" are only a part of the evaluation picture. Ve 
are the voices that keeps saying, "Data, not dogma." We are the purveyors of data, but ve 
are not THE EVALUATOSS. That role is shared. The remaining papers in this 
symposium will describe some of the other faces of research and evaluation in District 4J. 
It is my belief that the studies they present share the conceptual framework I outlined - 
that data should be used for informed decision-making, that results must be 
understandable, and that a cooperative atmosphere is the most productive one. 



